Method For Dynamically Reconfiguring Machine Learning Models

ABSTRACT

Systems and methods of reconfiguring machine learning models are disclosed. A method includes receiving, by a server computer, external data and determining, by the server computer, a machine learning model from a plurality of machine learning models, based on the external data. The plurality of machine learning models are in standby mode. The server computer configures an AI engine with the machine learning model. The method also includes receiving, by the server computer, a prediction request from a client computer, processing the prediction request with the AI engine to form a prediction response, and sending the prediction response to the client computer.

CROSS-REFERENCES TO RELATED APPLICATIONS

None.

BACKGROUND

Some machine learning models are designed to make predictions andanalyze things that are relatively constant time or in closely relatedcontexts. However, models might need to make predictions that span awide range of contexts. For example, a model can predict future behaviorof a user. The behavior may vary significantly depending on factors suchas time of year, vacations, recent major events, etc. Training a modelto be robust within a wide range of contexts can result in a model thatis too large or unwieldy.

Additionally, the contexts may be contradictory, and attempting to traina model with contradicting data may result in a model that isineffective with all contexts. For example, a model may be used toauthenticate a user based on the location of a log-in attempt. The modelmay be optimized to differentiate between log-in attempts while the useris at home and fraudulent log-in attempts. Another context that maycontradict the optimized model may be while the user is logging inremotely on vacation. Log-in attempts while the user is on vacation maybe legitimate, but may resemble fraudulent log-in attempts. While it maybe possible to train a model with the home data, vacation data, andfraud data this may be time-consuming and may result in increased falsepositives and/or false negatives since the model is not optimized for aparticular context.

Embodiments of the invention address these and other problemsindividually and collectively.

BRIEF SUMMARY

One embodiment of the invention includes receiving, by a servercomputer, external data and determining, by the server computer, amachine learning model from a plurality of machine learning models basedon the external data. The method also includes updating, by the servercomputer, a configuration database with the machine learning model andconfiguring, by the server computer, an AI engine with the machinelearning model using the configuration database. The method alsoincludes receiving, by the server computer, a prediction request from aclient computer, processing, by the server computer, the predictionrequest with the AI engine to form a prediction response, and sending,by the server computer, the prediction response to the client computer.

Another embodiment of the invention includes a server computercomprising a processor and a non-transitory computer-readable medium,comprising code executable by the processor for implementing a methodcomprising receiving external data and determining a machine learningmodel from a plurality of machine learning models based on the externaldata. The method also includes updating a configuration database withthe machine learning model and configuring an AI engine with the machinelearning model using the configuration database. The method alsoincludes receiving a prediction request from a client computer,processing the prediction request with the AI engine to form aprediction response, and sending the prediction response to the clientcomputer.

Further details regarding embodiments of the invention can be found inthe Detailed Description and the Figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a system and sequence diagram according to embodiments.

FIG. 2 shows a block diagram of a configuration service computeraccording to embodiments.

FIG. 3 shows a user interface for a configuration database according toembodiments.

FIG. 4A and FIG. 4B show an exemplary configuration database accordingto embodiments.

FIG. 5 shows an exemplary cached configuration database according toembodiments.

DETAILED DESCRIPTION

Embodiments of the invention can allow the use of a machine learningmodel for many different situations, by dynamically reconfiguring themachine learning model that is used to make predictions. Each of themachine learning models may be trained for a particular context or typeof prediction request. For example, one embodiment may include a set ofmodels, each trained for users in a different geographic location. Inembodiments, the machine learning models may be classification models,such as for determining whether or not a transaction is fraudulent, orwhether or not an attempt to log in to a secure data server islegitimate. Additionally, or alternatively, the machine learning modelsmay be used to predict a value, such as a total future sales at abusiness or future traffic to a website. Each of the machine learningmodels can be deployed in stand-by mode on an AI processing computer sothat it can be used when needed, based on the configuration of acomputer that runs the machine learning models. The configuration can bestored in a configuration database in a configuration service computer.

Applications that use the machine learning models can cache portions ofthe configuration database, such as portions relating to particularusers or machine learning models. When the application receives aprediction request to process with a machine learning model, theapplication can retrieve configuration information from the cache.Determining the configuration may be done by a rules system based onexternal data, prior to receiving a prediction request. By determiningthe configuration based on relevant external data, and on a user-by-user(or request-by-request) basis, the AI engine can be responsive tovarious requests. Embodiments thus address the difficulties that canexist with a large model that attempts to encompass everything in asingle model.

Prior to discussing embodiments of the invention, some terms can bedescribed in further detail.

A “server computer” is typically a powerful computer or cluster ofcomputers. For example, the server computer can be a large mainframe, aminicomputer cluster, or a group of servers functioning as a unit. Inone example, the server computer may be a database server coupled to aWeb server.

A “processor” may include any suitable data computation device ordevices. A processor may comprise one or more microprocessors workingtogether to accomplish a desired function. The processor may includeCPUs comprising at least one high-speed data processor adequate toexecute program components for executing user and/or system-generatedrequests. The CPU may be a microprocessor such as AMD's Athlon, Duronand/or Opteron; IBM and/or Motorola's PowerPC; IBM's and Sony's Cellprocessor; Intel's Celeron, Itanium, Pentium, Xeon, and/or XScale;and/or the like processor(s).

A “memory” may be any suitable device or devices that can storeelectronic data. A suitable memory may comprise a non-transitorycomputer readable medium that stores instructions that can be executedby a processor to implement a desired method. Examples of memories maycomprise one or more memory chips, disk drives, etc. Such memories mayoperate using any suitable electrical, optical, and/or magnetic mode ofoperation.

A “user device” may be any suitable electronic device that can processand communicate information to other electronic devices. The user devicemay include a processor and a computer-readable medium coupled to theprocessor, the computer-readable medium comprising code, executable bythe processor. The user device may also each include an externalcommunication interface for communicating with each other and otherentities. Examples of user devices may include a mobile device, a laptopor desktop computer, a wearable device, etc.

A “user” may include an individual or a computational device. In someembodiments, a user may be associated with one or more user devicesand/or personal accounts. In some embodiments, the user may be acardholder, account holder, or consumer.

A “machine learning model” may include an application of artificialintelligence that provides systems with the ability to automaticallylearn and improve from experience without explicitly being programmed. Amachine learning model may include a set of software routines andparameters that can predict an output of a process (e.g., identificationof an attacker of a computer network, authentication of a computer, asuitable recommendation based on a user search query, etc.) based on a“feature vector” or other input data. A structure of the softwareroutines (e.g., number of subroutines and the relation between them)and/or the values of the parameters can be determined in a trainingprocess, which can use actual results of the process that is beingmodeled, e.g., the identification of different classes of input data.Examples of machine learning models include support vector machines,which are models that classify data by establishing a gap or boundarybetween inputs of different classifications, and neural networks, whichare collections of artificial “neurons” that perform functions byactivating in response to inputs.

A “machine learning classifier” may include a machine learning modelthat can classify input data or feature vectors. For example, an imageclassifier is a machine learning model that can be used to classifyimages, such as images of animals. As another example, a news classifieris a machine learning model that can classify news articles as “realnews” or “fake news.” As a third example, an anomaly detector, such as acredit card fraud detector, can classify input data such as credit cardtransactions as either normal or anomalous. The output produced by amachine learning classifier may be referred to as “classification data.”Machine learning classifiers may also include clustering models, such asK—means clustering. Clustering models can be used to partition inputdata or feature vectors into multiple clusters. Each cluster maycorrespond to a particular classification. For example, a clusteringmodel may accept feature vectors corresponding to the size and weight ofdogs, then generate clusters of feature vectors corresponding to smalldogs, medium dogs, and large dogs. When new input data is included in acluster (e.g., the small dogs cluster), the clustering model haseffectively classified the new input data as input data corresponding tothe cluster.

An “AI engine” may comprise a plurality of models with artificialintelligence capabilities. Each model may perform tasks relating to oneor more different areas of artificial intelligence, such as machinelearning, natural language processing, and knowledge representation. TheAI engine may be able to integrate the models, for example, to use thenatural language processing to process a question and then use machinelearning to generate a prediction in response to the question.

A “prediction request” may be a request for a predicted answer to aquestion. For example, a prediction request may be a request for someinformation about a future event, a classification prediction, anoptimization, etc. The prediction request may be in the form of naturallanguage (e.g., as an English sentence) or may be in a computer-readableformat (e.g., as a vector). The answer to a prediction request may ormay not be determined using artificial intelligence.

“Contextual data” may include data that provides circumstancessurrounding an event, entity, or item. Contextual data may be additionalinformation that gives a broader understanding or more detail aboutexisting information or requests (e.g., a prediction request). Inembodiments, contextual data may include transaction histories,geographic data, census data, etc.

“External data” may include data that provides circumstances surroundingan event, entity, or item. External data may be additional informationthat gives a broader understanding or more detail about a user orenvironment. In embodiments, external data may include transactionhistories, geographic data, census data, etc.

A “cache” may be a hardware or software component that stores data tofacilitate future retrieval of the data. The cache may be auxiliarymemory or storage that can store a subset of a larger database.

A “configuration database” may be a database that stores configurationinformation for a software or hardware component. A “configuration” mayinclude a set of information a software or hardware component. Inembodiments, a configuration database may include information about amodel, users, versions, software engines, etc.

An “in-memory database” may be a database that is stored in main memoryof a computer. This may be in contrast to a database that is stored ondisk storage of the computer. An in-memory database may allow for fastretrieval of data in the database. Some in-memory databases may store acopy of some or all of the database on disk storage. Examples ofin-memory database software can include Redis, Memcached, and MonetDB.

A “key-value database” may be a database that stores data with a keythat uniquely identifies a value that comprises the data. The value in akey-value database may be any type of data, such as an integer, astring, a date, etc. and any combination of data types. New keys andvalues can be added without impacting the rest of the database. This maybe in contrast to a relational database, where the data structure anddata types are predefined. Examples of key-value database software mayinclude Redis, ArangoDB, and Dynamo.

“Standby mode” can describe a mode of a machine learning model. Amachine learning model that is in standby mode can be a machine learningmodel that is deployed, but not being used. A deployed model may be inproduction and fully integrated into a system, and thus capable of beingused by the system to generate predictions. A model that is in standbymode may be integrated into the system but may not receive inputs toprocess.

FIG. 1 shows a block diagram of a system according to embodiments. Thesystem comprises a user device 110, a data enrichment computer 120, anAI processing computer 130, a rule processing computer 140, and aconfiguration service computer 150. Any of the devices in FIG. 1 may bein communication via a suitable communication network.

The communication network may include any suitable communication medium.The communication network may be one and/or the combination of thefollowing: a direct interconnection; the Internet; a Local Area Network(LAN); a Metropolitan Area Network (MAN); an Operating Missions as Nodeson the Internet (OMNI); a secured custom connection; a Wide Area Network(WAN); a wireless network (e.g., employing protocols such as, but notlimited to a Wireless Application Protocol (WAP), I-mode, and/or thelike); and/or the like. Message between the entities, providers,networks, and devices illustrated in FIG. 1 may be transmitted using asecure communications protocols such as, but not limited to, FileTransfer Protocol (FTP); HyperText Transfer Protocol (HTTP); SecureHypertext Transfer Protocol (HTTPS), Secure Socket Layer (SSL), ISO(e.g., ISO 8583) and/or the like.

User device 110 can be a computer that sends prediction requests to beprocessed by a machine learning model. The user device 110 may be, forexample, a laptop, a desktop computer, a smartphone, a server computer,etc. In some embodiments, the user device 110 may be operated by a userand may be a personal computer. In other embodiments, user device 110may be a server computer that is part of a computer network. Forexample, the user device 110 may be part of a payment processingnetwork.

Data enrichment computer 120 can add contextual data to a predictionrequest to ensure the prediction request has the correct information tobe processed by the AI processing computer. For example, contextual datamay include a timestamp, transaction data, geographic data, etc. Thecontextual data that is added may depend on the machine learning model,or the configuration. The configuration service computer 150 may sendinformation about the contextual data needed for the machine learningmodel to the data enrichment computer 120. Additionally, oralternatively, the data enrichment computer 120 may query aconfiguration table to determine the contextual data needed for aprediction request.

AI processing computer 130 may be a computer that executes machinelearning models using an AI engine. AI processing computer 130 may usethe AI engine to operate a plurality of machine learning models. In someembodiments, the machine learning models may be in standby mode on theAI processing computer 130. In standby mode, a machine learning modelmay be trained and deployed, but may not be active. In some embodiments,the AI processing computer 130 may only have one active machine learningmodel at a time. The active machine learning model may be the machinelearning model that the AI engine uses to process a received predictionrequest.

Rule processing computer 140 may determine a machine learning model froma plurality of machine learning models based on external data. Externaldata may include, for example, transaction data, location data, seasonaldata, etc. The external data may be received from sources such as publicdata sources (e.g., news sources, weather reports) and protected userdata (e.g., payment processing networks, user location databases). Insome embodiments, the rule processing computer 140 can apply rules tothe external data to determine the machine learning model. For example,the external data may include transaction data indicating that a userpurchased an airplane ticket and a hotel room. This may trigger a rulethat the user is likely on vacation and the rule processing computer 140may determine that a vacation machine learning model is appropriate. Asanother example, the external data may be a location of a user and adate, indicating that there is an upcoming holiday in the location ofthe user. For example, in the United States, users may have differentspending habits in the “holiday season” of November and December. Thus,rule processing computer 140 may determine a holiday machine learningmodel after receiving external data indicating that a user is in theUnited States and that it is during the holiday season.

Configuration service computer 150 may maintain a configuration databasefor configuring machine learning models. The configuration servicecomputer 150 can maintain an in-memory database that stores theconfiguration database. The configuration service computer 150 may alsocache the configuration database to external storage. The configurationservice computer 150 may receive information about a machine learningmodel from the rule processing computer 140, then update theconfiguration database based on the machine learning model. For example,the configuration service computer 150 may update the configurationdatabase to add a model or disable a currently enabled model.

FIG. 2 shows a block diagram of a configuration service computer 150.Configuration service computer 150 comprises a processor 152, a memory154, a network interface 156, and a computer-readable medium 158. Thecomputer-readable medium 158 can comprise a configuration determinationmodule 158A, a table update module 158B, a communication module 158C,and an in-memory configuration database 158D. The computer may alsocomprise a configuration database storage 153

The processor 152 may be implemented as one or more integrated circuits(e.g., one or more single core or multicore microprocessors and/ormicrocontrollers). The processor 152 may be used to control theoperation of the configuration service computer 150. The processor 152can execute a variety of programs in response to program code orcomputer-readable code stored in memory 154. The processor 152 mayinclude functionality to maintain multiple concurrently executingprograms or processes.

The memory 154 may be implemented using any combination of any number ofnon-volatile memories (e.g., flash memory) and volatile memories (e.g.,DRAM, SRAM), or any other non-transitory storage medium, or acombination of media. Memory 154 may store the in-memory configurationdatabase 158D.

Network interface 156 may be configured to connect to one or morecommunication networks to allow configuration service computer 150 tocommunicate with other entities such as rule processing computer 140, AIprocessing computer 130, etc. For example, communication with the ruleprocessing computer 140 can be direct, indirect, and/or via an API.

Computer-readable medium 158 may comprise one or more non-transitorymedia for storage and/or transmission. Suitable media include, asexamples, a random access memory (RAM), a read only memory (ROM), amagnetic medium such as a hard-drive, or an optical medium such as a CD(compact disk) or DVD (digital versatile disk), flash memory, and thelike. The computer-readable medium 158 may be any combination of suchstorage or transmission devices.

Configuration determination module 158A, in conjunction with theprocessor 152, can determine a configuration that corresponds to aparticular machine learning model. Configuration determination module158A may include a table or database with configuration information fora plurality of models. For example, configuration determination module158A may receive an indication to use Model A for user 1234.Configuration determination module 158A can then determine aconfiguration for Model A (e.g., a machine learning model version, asoftware engine, etc.). In some embodiments, configuration determinationmodule 158A may determine that the configuration database alreadyincludes configuration information for Model A to be used for user12134, and thus the configuration database should be updated to enableModel A for user 1234.

Table update module 158B, in conjunction with the processor 152, canupdate the configuration table. The table update module 158B may receiveconfiguration information from the configuration determination module158A. The table update module 158B may access the configuration databasestorage 153. Additionally, the table update module 158B may access thein-memory configuration database 158D. In some embodiments, table updatemodule 158B may check that only one model is active for a user at atime.

Communication module 158C, in conjunction with the processor, maycommunicate configuration information to other devices, such as dataenrichment computer 120 and AI processing computer 130. In someembodiments, the communication module may send the configurationinformation directly to the other computers. In other embodiments, thecommunication module 158C can send an indication to the other computersthat the configuration database has been updated.

In-memory configuration database 158D may be a database storingconfiguration information (e.g., machine learning model version number,software engine, etc.) about a plurality of machine learning models. Thein-memory configuration database 158D may be a key-value in-memorydatabase such as Redis.

Configuration database storage 153 may be an external storage of theconfiguration database. The configuration database storage 153 may be acache of the in-memory configuration database 158D. The configurationdatabase may be stored in the configuration database storage 153, forexample, every few seconds, every few minutes, or every few seconds.Configuration database storage 153 may be a hard disk drive, asolid-state drive, an optical storage device (e.g., a CD or DVD drive),or the like.

Returning to FIG. 1, a process flow can be described.

In step S102, rule processing computer 140 can receive external datafrom one or more external data sources. The external data may comprisetransaction data, temporal data, etc. The external data (not shown) maybe received from sources such as public data sources (e.g., newssources, weather reports) and protected user data (e.g., paymentprocessing networks, user location databases). In some embodiments, therule processing computer 140 may receive the external data continuouslyor periodically. For example, the external data from one source may bereceived every hour or every day. Other external data may be receivedsporadically. For example, the rule processing computer 140 may receivetransaction data for a particular user from a payment processing networkwhen the user conducts a transaction. In some embodiments, the externaldata may be associated with a particular user identifier (e.g., a PAN, aphone number, an email address). As an example, rule processing computer140 may receive information that a user has purchased a plane ticket andreserved a hotel room, including the PAN of the user.

In step S104, rule processing computer 140 can determine a machinelearning model from a plurality of machine learning models by applyingrules to the external data. For example, the rule processing computer140 may have a rule that Model A should be enabled for all users in aparticular country. As another example, the rule processing computer 140may have a rule that Model B (e.g., a vacation fraud detection model)should be enabled for users that have recently made a purchase at aparticular merchant (e.g., hotel, airline). The plurality of machinelearning models may comprise different machine learning models. Forexample, the plurality of machine learning models may include deeplearning models (e.g., support vector machine, recurrent neural network)and business rule models. Additionally, or alternatively, the pluralityof machine learning models may include different versions of one or moremachine learning models. For example, one version of a model may be afraud prediction model for a user in summer and another version may be afraud prediction model for a user in winter. Therefore, determining themachine learning model may include determining the model and theversion.

After determining a machine learning model, the rule processing computer140 can send an indication to the configuration service computer 150about the determined machine learning model. The indication may includeinformation such as a name and version of the machine learning model. Insome embodiments, the rule processing computer 140 can determine themachine learning model on an individual user basis. For example, ruleprocessing computer 140 may determine that Model B (e.g., a vacationfraud detection model) should be enabled for user 1234 because user 1234has recently made a purchased a plane ticket and hotel reservation. Thenthe indication may also include the user identifier. In otherembodiments, the rule processing computer 140 can determine the machinelearning model for a plurality of users. For example, if the ruleprocessing computer 140 determines that it is a holiday season in aparticular geographic region, then the rule processing computer 140 maydetermine a holiday machine learning model for all users in thegeographic region. For example, a “holiday” machine learning model maybe determined for all users in the United States during November andDecember.

In step S106, the configuration service computer 150 can configure an AIengine with the machine learning model by updating a configurationdatabase. The configuration database may be a an in-memory database thatstores configuration data for executing machine learning models on an AIengine of the AI processing computer 130. In updating the configurationtable, the configuration service computer 150 can change values in theconfiguration table to enable or disable particular machine learningmodels. In some embodiments, there may be a value in the configurationtable to indicate that a machine learning model is enabled or disabled.There may also be a value indicating a particular version of the machinelearning model (e.g., version 2.7 or version 3.1).

In some embodiments, the configuration service computer 150 may updatethe configuration table for a particular user. The configuration servicecomputer 150 may use the user identifier when updating the configurationtable. Alternatively, the configuration service computer 150 may updatethe configuration table for a plurality of users or range of useridentifiers, such as all users in a geographic area.

In step S108, the configuration service computer 150 can send anindication of the machine learning model to data enrichment computer120. Additionally, or alternatively, the configuration service computer150 can send an indication that the configuration database has beenupdated. The data enrichment computer 120 can use the indication of themachine learning model to determine what contextual data to add to aprediction request. In some embodiments, the data enrichment computer120 can query the configuration table to determine what contextual datato add.

In step S110, the configuration service computer 150 can send anindication of the machine learning model to AI processing computer 130.Additionally, or alternatively, the configuration service computer 150can send an indication that the configuration database has been updated.In some embodiments, the AI processing computer 130 may periodicallycache the configuration database. When the AI processing computer 130caches the configuration database with updated configurationinformation, the AI processing computer 130 may treat this as anindication of the machine learning model. For example, if the machinelearning models are neural networks, the AI engine may store matriceswith weights of neurons for the plurality of machine learning models.The configuration information may indicate which matrix of weights (andthus which neural network) should be used. In some embodiments, the AIengine may have a plurality of machine learning models in standby mode.Machine learning models may be models that are deployed and integratedinto a prediction generation system but may not be actively used ingenerating predictions.

In step S112, user device 110 can send a prediction request to dataenrichment computer 120. The prediction request can be a request for aprediction or classification from a machine learning model. For example,a prediction request can be a fraud prediction for a transaction or alog-in attempt. In a transaction example, the prediction request maycomprise an indicator that it is a fraud request, a user identifier, atransaction amount, a merchant identifier, a location, and a time stamp.In a log-in example, the prediction request may comprise an indicatorthat it is an authorization request, a user identifier, a deviceidentifier, a password (or a derivative of the password), a location,and a time stamp. As an example, user device 110 may send anauthorization request to data enrichment computer 120 for a transactionthat a user is making while on vacation. In some embodiments, the userdevice 110 can be a computer operated by a user, such as a laptop orsmartphone. The prediction request may be generated, for example, by theuser typing in a question or speaking a question. The prediction requestmay be generated, for example, in response to receiving anauthentication or authorization request message.

In step S114, data enrichment computer 120 can enrich the predictionrequest with contextual data. Contextual data may include, for example,transaction data or geographic data and may be received from, forexample, a location service or a payment processing network. Forexample, if the prediction request is an authorization request for auser that is on vacation, the data enrichment computer 120 may addinformation about the past three transactions that were made by theuser. Some or all of the contextual data added by the data enrichmentcomputer 120 may be dependent on the machine learning model. As anexample, the prediction request for a fraud prediction may not haveincluded a location of the interaction, but the machine learning modelmay require a location as input for the fraud prediction. The dataenrichment computer 120 may then determine a location to enrich thefraud prediction.

In some embodiments, the data enrichment computer 120 can use theindication of the machine learning model to determine the contextualdata to add. In other embodiments, the data enrichment computer 120 mayuse the configuration database to determine the contextual data to add.For example, the data enrichment computer 120 may query theconfiguration database to identify the determined machine learningmodel. Additionally, or alternatively, the configuration database mayinclude values indicating the appropriate contextual data, and the dataenrichment computer 120 may query the configuration table to determinethe contextual data needed. For example, the configuration database mayindicate that a timestamp should be added to a query. Data enrichmentcomputer 120 can also format the prediction request into an appropriateformat for the AI processing computer 130.

In step S116, data enrichment computer 120 can send the enrichedprediction request to AI processing computer 130. The AI processingcomputer 130 can then process the prediction request with the machinelearning model on the AI engine. For example, if the prediction requestis an authorization request for a user that is on vacation, the AIprocessing computer 130 can determine whether or not the transaction isfraudulent with a vacation transaction fraud model. This may be adifferent fraud model than might have been used if the user was at home.The AI processing computer 130 can determine the machine learning modelfrom the configuration table. The configuration table may also includeother information for configuring the machine learning model.

The AI processing computer 130 can then generate a prediction responsewith the machine learning model. For example, if the prediction requestis a fraud prediction request for a transaction, the AI processingcomputer 130 can return a prediction response indicating whether to ornot the transaction is fraudulent. In some embodiments, the predictionresponse may be a classification (e.g., fraud or not fraud). In otherembodiments, the prediction response may be a value (e.g., predictedtraffic to a website).

In step S118, AI processing computer 130 can return the predictionresponse to the user device 110. For example, AI processing computer 130can return an authorization response message to the user device 110. Insome embodiments, the AI processing computer 130 can return theprediction response to the user device 110 directly. In someembodiments, the AI processing computer 130 can return the predictionresponse to the user device 110 through data enrichment computer 120.

FIG. 3 shows an exemplary configuration table according to embodiments.This may be a user interface for viewing or modifying the configurationdatabase. In some embodiments, the user interface may display a cache ofthe configuration database, and changes made to the configurationdatabase through the user interface may be sent to the configurationservice computer to update the configuration database. The userinterface may be presented through the configuration service computer,the AI processing computer, or another computer with access to theconfiguration database.

Block 305 may be a menu for selecting an application. Differentapplications can use the configuration database. Each application canhave an AI processing computer that uses a saved configuration table.For example, the application may be Visa Dynamic Network Analysis.

Block 315 may be a menu for selecting a particular configuration toview. An application may have multiple configurations available. As anexample, this configuration can be displayed as product_model_config.After a configuration is selected, it can be displayed in the table 300.

Table 300 shows a table of configuration information. The table hascolumns for configuration information for each user. New rows can beadded to the table, and existing rows can be edited or deleted. Thetable may be built from a plurality of values of a configurationdatabase. For example, each row of the configuration table may be storedas a separate value in the configuration database.

Column 310 shows actions that can be performed on the data in the table.There may be an option to enter data for a new row. In existing rows,there may be an option to edit the data and an option to delete the row.

Column 320 shows user identifiers, such as primary account numbers(PANs) of users. Other embodiments may use other user identifiers (e.g.,a phone number, email, merchant identifier, address, etc.). In someembodiments, column 320 may display a value derived from a useridentifier (e.g., a hash, a token) to protect user information. Someuser identifiers may appear in multiple rows (such as 4123111111111111in the exemplary table), which may indicate that there are multiplepossible configurations available for that user.

Column 330 shows an indication of whether or not the machine learningmodel is enabled. The column may contain values such as “true” and“false”, “0” and “1”, or “enabled” and “disabled”. The configurationdatabase may use uniqueness checking to ensure that only one model isenabled for a particular user identifier at a time.

Column 340 shows the software engine that can run the machine learningmodel. The software engine may be deep learning software, such asTensorFlow. Deep learning software may be used for running machinelearning models. Additionally, or alternatively, the software engine maybe rules engine software such as OpMesh. A rules engine may be used forrunning a business rules model.

Column 350 shows the name of the machine learning model. For example,models can include VCAS and VCO. Some models may be location specific,such as “default_EU” and “default_UK”.

Column 360 shows the version of the machine learning model. Each modelmay have one or more versions. In some embodiments, later versions of amachine learning model may indicate an improvement in quality over pastmodels. In other embodiments different versions of the model may bedeveloped for a variety of situations. For example, version 2.4 of afraud prediction model may be trained for a user in their home region.Version 2.5 of the same fraud prediction model may be trained for theuser when the user is on vacation.

FIG. 4A and FIG. 4B show a configuration database of a configurationservice computer. The configuration database may be an in-memorykey-value database. The configuration database can be stored in thememory of the configuration service computer. The configuration databasemay be periodically backed up to disk storage of the configurationservice computer. For example, the configuration database may be backedup every few seconds, every few minutes, or every hour. In someembodiments, the entire configuration database may be backed up. Inother embodiments, the configuration service computer may maintain arecord of changes that are made to the configuration database and mayperiodically update the stored database with the changes.

The configuration database may be a NoSQL database, such as Redis. Redisis a key-value database, and can store data types other than (andincluding) strings. Table 410 is a table with some key-value pairs. Onekey may be product_model_config, which maps to a hash map of otherinformation. Another key in the table 410 is ETDC_AppNames, which mapsto a table 420 that links to information about the applications, likeApp1 which has information about the application.

Table 410 may also include a hashmap containing the configurationinformation. The key of table 410 have a key for a particular machinelearning model, such as product_model_config. The value of table 410 maybe another table 430. Table 430 may be a hashmap. One element of table430 may be a list of user identifiers (e.g., PANs) in table 440. Eachuser identifier in 430 can have a value with the configurationinformation. There may be multiple values associated with the useridentifier, such as table 450 and 460. Each of the tables 450, 460 mayinclude configuration information for a particular machine learningmodel.

Table 450 comprises a key for user identifier, whether or not the modelis enabled, the score engine, the model name, and the model version.Both table 450 and table 460 can be associated with the same key of theuser identifier and thus may have similar values.

FIG. 5 shows a cache of the configuration database. In some embodiments,the cache can be a Guava cache. Other embodiments can use other cacheprograms. The cache may include a portion of the configuration database.The portion may include configuration information relating to particularusers, machine learning models, or applications.

When the portion of the database is cached, it can be cached to anapplication computer. The application computer might be the AI servicecomputer, if the AI service is a particular application. The databasemay be cached to the application computer periodically. For example, theapplication computer may update the cache every hour. Additionally, oralternatively, the application computer may receive an indication thatthe database has been updated and then cache the database.

The cached database can comprise a table 510. The table 510 may be asubset of the table 410 of FIG. 4. The parts of 410 that are transferredto table 510 may be the elements of 410 that relate to a particularmachine learning model configuration, such as product_model_config. Notsure how much of the table is also transferred.

All data related to a particular machine learning model configuration(e.g., product_model_config) can be cached. For example, table 530 maybe table 430, table 540 may be table 440, table 550 and 550 may betables 460 and 460, respectively. The tables may be static, and mayrepresent the makeup of the corresponding tables at the time they werecached.

Embodiments of the invention provide a number of advantages. Embodimentscan allow the use of several machine learning models to addressdifferent situations that might be used to process data and generatepredictions. Embodiments allow a plurality of models to be trained, eachspecialized for a particular situation and context. By deploying eachmodel in standby mode, they can be prepared without being active.

Embodiments can allow responsive model usage. In some instances, theprocess of developing, training, and deploying a new machine learningmodel can take several months. By the time a new model is developed fora situation, the situation may no longer be relevant. Creating one modelthat encompasses all situations can result in a model that is verylarge. Additionally, the model may not even be able to be optimized withtoo many different types of training data. Instead of developing amachine learning model for a particular context and hoping that themodel is broad enough to cover future changes, the AI processingcomputer can easily change machine learning models, in a matter ofminutes or seconds instead of months, to respond to changes as theyarise. This may also allow for predicting future user actions, such aspredicting that a user will go on vacation and enabling an appropriatemodel for that user. Enabling computers to cache the configurationdatabase can allow wider use of the configuration information. Inembodiments, a central computer (or group of computers) can determineappropriate models and then applications that use those models can cachethe configuration database so that they can retrieve the configurationinformation as needed

Any of the software components or functions described in thisapplication, may be implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java, C++ or Perl using, for example, conventional or object-orientedtechniques. The software code may be stored as a series of instructions,or commands on a computer readable medium, such as a random accessmemory (RAM), a read only memory (ROM), a magnetic medium such as ahard-drive, or an optical medium such as a CD-ROM. Any such computerreadable medium may reside on or within a single computationalapparatus, and may be present on or within different computationalapparatuses within a system or network.

The above description is illustrative and is not restrictive. Manyvariations of the invention may become apparent to those skilled in theart upon review of the disclosure. The scope of the invention can,therefore, be determined not with reference to the above description,but instead can be determined with reference to the pending claims alongwith their full scope or equivalents.

One or more features from any embodiment may be combined with one ormore features of any other embodiment without departing from the scopeof the invention.

A recitation of “a”, “an” or “the” is intended to mean “one or more”unless specifically indicated to the contrary.

All patents, patent applications, publications, and descriptionsmentioned above are herein incorporated by reference in their entiretyfor all purposes. None is admitted to be prior art.

1-20. (canceled)
 21. A method comprising: training, by a servercomputer, a plurality of machine learning models, wherein each machinelearning model is trained with respect to a context of a plurality ofcontexts; generating, by the server computer, a configuration table tobe associated with a user, the configuration table including: (i) a hashmap that links an identifier of the user to the plurality of machinelearning models, and (ii) configuration information associated with eachof the plurality of machine learning models, the configurationinformation being required for executing the machine learning model;receiving, by the server computer, a request from a client computeroperated by the user, the request being associated with a first contextof the plurality of contexts; selecting, by the server computer, a firstmachine learning model from the plurality of machine learning models;configuring, by the server computer, an AI engine with the configurationinformation corresponding to the machine learning model obtained fromthe configuration table; processing, by the server computer, based onthe configuration information, the request with the AI engine to form aresponse; and sending, by the server computer, the response to theclient computer.
 22. The method of claim 21, wherein the server computerselects the first machine learning model based on a second contextassociated with the first machine learning model matching the firstcontext associated with the request.
 23. The method of claim 21, furthercomprising: receiving, by the server computer, external data from one ormore public data sources, the external data including informationrelated to the user; and determining, by the server computer, the firstmachine learning model to be used to generate the response based on theexternal data.
 24. The method of claim 23, wherein the external dataincludes data related to an event occurring in an environment of theuser.
 25. The method of claim 23, wherein the external data includesdata indicating that the user is in a particular geographic region. 26.The method of claim 21, wherein the plurality of machine learning modelscomprises different machine learning models.
 27. The method of claim 21,wherein the plurality of machine learning models comprises differentversions of the first machine learning model.
 28. The method of claim21, wherein the configuration information includes a type of AI engineused to implement the first machine learning model, a name of the firstmachine learning model, a version of the first machine learning model,and a status indicator indicating an operational status of the firstmachine learning model.
 29. The method of claim 28, further comprising:maintaining, by the server computer, the plurality of machine learningmodels in a standby mode; and updating the status indicator of the firstmachine learning model based on the selecting.
 30. The method of claim31, wherein the hash map links the identifier of the user to the firstmachine learning model and a second machine learning model, and whereina first configuration information associated with the first machinelearning model is different from a second configuration informationassociated with the second machine learning model.
 31. A server computercomprising: a processor; and a non-transitory computer-readable medium,comprising code executable by the processor for implementing a methodcomprising: training, by the server computer, a plurality of machinelearning models, wherein each machine learning model is trained withrespect to a context of a plurality of contexts; generating aconfiguration table to be associated with a user, the configurationtable including: (i) a hash map that links an identifier of the user tothe plurality of machine learning models, and (ii) configurationinformation associated with each of the plurality of machine learningmodels, the configuration information being required for executing themachine learning model; receiving a request from a client computeroperated by the user, the request being associated with a first contextof the plurality of contexts; selecting a first machine learning modelfrom the plurality of machine learning models; configuring an AI enginewith the configuration information corresponding to the machine learningmodel obtained from the configuration table; processing based on theconfiguration information, the request with the AI engine to form aresponse; and sending the response to the client computer.
 32. Theserver computer of claim 31, wherein the server computer selects thefirst machine learning model based on a second context associated withthe first machine learning model matching the first context associatedwith the request.
 33. The server computer of claim 31, whereinconfiguring the AI engine comprises changing one or more of theplurality of machine learning models, a version of a particular machinelearning model, and/or a software engine used to execute the particularmachine learning model.
 34. The server computer of claim 31, wherein thehash map links the identifier of the user to the first machine learningmodel and a second machine learning model, and wherein a firstconfiguration information associated with the first machine learningmodel is different from a second configuration information associatedwith the second machine learning model.
 35. The server computer of claim31, wherein the processor is further configured for: receiving externaldata from one or more public data sources, the external data includinginformation related to the user; and determining the first machinelearning model to be used to generate the response based on the externaldata.
 36. The server computer of claim 31, wherein the configurationinformation includes a type of AI engine used to implement the firstmachine learning model, a name of the first machine learning model, aversion of the first machine learning model, and a status indicatorindicating an operational status of the first machine learning model.37. The server computer of claim 36, wherein the processor is furtherconfigured for: maintaining, by the server computer, the plurality ofmachine learning models in a standby mode; and updating the statusindicator of the first machine learning model based on the selecting.38. The server computer of claim 31, the a portion of the configurationinformation is cached.
 39. The server computer of claim 38, wherein theportion of the configuration information is cached in an AI processingcomputer, and wherein the AI processing computer comprises the AIengine.
 40. The server computer of claim 31, wherein the request is afraud detection request or a secure log-in request.